Feature Selection for Designing a Novel Differential Evolution Trained Radial Basis Function Network for Classification

This work presents a novel approach for classification of both balanced and unbalanced dataset by suitably tuning the parameters of radial basis function networks with an additional cost of feature selection. Inputting optimal and relevant set of features to a radial basis function may greatly enhance the network efficiency (in terms of accuracy) at the same time compact it size. In this paper, the authors use information gain theory (a kind of filter approach) for reducing the features and differential evolution for tuning center and spread of radial basis functions. The proposed approach is validated with a few benchmarking highly skewed and balanced dataset retrieved from University of California, Irvine (UCI) repository. The experimental study is encouraging to pursue further extensive research in highly skewed data. DOI: 10.4018/jamc.2013010103 International Journal of Applied Metaheuristic Computing, 4(1), 32-49, January-March 2013 33 Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. if the whole dataset is used. Many variants of evolutionary and non-evolutionary based approaches are discussed in Derrac, Garcia, and Herrera (2010). The ideal outcome of instance selection is a model independent, minimum sample of data that can accomplish tasks with little or no performance deterioration. However, in this work, we restrict ourselves with feature selection only. Feature selection can be broadly classified into two categories: 1. filter approach (it depends on generic statistical measurement); and 2. wrapper approach (based on the accuracy of a specific classifier) (Aruna et al., 2012). In this work, the feature selection is performed based on information gain theory (entropy) measure with a goal to select a subset of features that preserves as much as possible the relevant information found in the entire set of features. After selection of the relevant set of features the fine tuned radial basis function network is modeled using differential evolution for classification. Over the decade radial basis function (RBF) networks have attracted a lot of interest in various domain of interest (Haykin, 1994; Novakovic, 2011; Naveen, Ravi, Rao, & Chauhan, 2010; Liu, Mattila, & Lampinen, 2005). One reason is that they form a unifying link between function approximation, regularization, noisy interpolation, classification, and density estimation. It is also the case that training RBF networks is usually faster than training multi-layer perceptron networks. RBF network training usually proceeds in two steps: First, the basis function parameters (corresponding to hidden units) are determined by clustering. Second, the final-layer weights are determined by least square which reduces to solving a simple linear system. Thus, the first stage is an unsupervised method which is relatively fast, and the second stage requires the solution of a linear problem, which is also fast. One of the advantages of RBF neural networks, compared to multi-layer perceptron networks, is the possibility of choosing suitable parameters for the units of hidden layer without having to perform a non-linear optimization of the network parameters. However, the problem of selecting the appropriate number of basis functions remains a critical issue for RBF networks. The number of basis functions controls the complexity, and hence the generalization ability of RBF networks. An RBF network with too few basis functions gives poor predictions on new data, i.e. poor generalization, since the model has limited flexibility. On the other hand, an RBF network with too many basis functions also yields poor generalization since it is too flexible and fits the noise in the training data. A small number of basis functions yields a high bias, low variance estimator, whereas a large number of basis functions yields a low bias but high variance estimator. The best generalization performance is obtained via a compromise between the conflicting requirements of reducing bias while simultaneously reducing variance. This trade-off highlights the importance of optimizing the complexity of the model in order to achieve the best generalization. However, choosing an optimal number of kernels is beyond the focus of this paper. In training procedure of RBFNs revealing center of gravity and width is of particular importance of the improvement of the performance of the networks. There are many approaches along the line with their own merits and demerits (Storn & Price, 1995; Storn & Price, 1997; Price, Storn, & Lampinen, 2005). This paper discusses the use of differential evolution to reveals hidden centers and spreads. The motivation using differential evolution (DE) over other EAs (Michalewicz, 1996) such as GAs (Goldberg, 1989) is that in DE string encoding are typically represented as real valued vectors, and the perturbation of solution vectors is based on the scaled difference of two randomly selected individuals of the current population. Unlike GA, the resulting step size and orientation during the perturbation process automatically adopt to the fitness function landscape. The justification behind combining the idea of feature selection with classification is to reduce the space, time, and accuracy. This article is set out as follows. Section 1 gives overview of RBF network, feature selection, and differential evolution. In Section 2, the 16 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the product's webpage: www.igi-global.com/article/feature-selection-designing-noveldifferential/77298?camid=4v1 This title is available in InfoSci-Journals, InfoSci-Journal Disciplines Computer Science, Security, and Information Technology. Recommend this product to your librarian: www.igi-global.com/e-resources/libraryrecommendation/?id=2